The present invention relates to an endoscopic system of the type which includes at least one endoscope having an objective which generates an image of an area to be observed, at least one video recorder which records the image generated by the objective, at least one instrument, a position-sensing device which detects a position of the at least one instruments, and an assessment and control unit to which an output signal of the video recorder and the position-sensing device are applied and which displays on a monitor the image and symbols representing the position of the instrument.
The endoscopic systems of the claimed general type may be real systems for the application on the human or animal body or in the engineering field but they may also be simulation or training systems which a physician can use, for instance, for training endoscopic operations.
In minimal-invasive surgery (MIC) for instance surgical operations are performed with video assistance. When the endoscope or endoscopes is/are introduced together with the associated endoscopic camera system and the instrument or instruments, the physician does initially not dispose of the visual orientation of the instrument or instruments relative to the endoscope(s). This is true at least as long as the instruments are not visible in the coverage field of the endoscope or endoscopes.
But even when the physician sees the instrument with the endoscope he or she obtains only a two-dimensional image so that the information about the orientation of the instrument or the position of the distal end of the instrument relative to the focal plane, respectively, is initially missing.
A similar situation prevails with the endoscopic training or simulation systems mentioned already. Advanced systems merely comprise a housing which simulates the patient""s body. Operating elements or handles, respectively, are mounted on the housing, which correspond to the proximal elements of the endoscopes or the handles of the instruments, respectively. The movements which the physician performs with the operating or control elements or handles, respectively, are detected by means of position sensors and supplied to a computer. The computer displays on a screen the movement of the distal functional elements, which do not exist in reality in advanced simulation systems, in an xe2x80x9capparently realxe2x80x9d environment, i.e. the respective body cavity, for instance, with a simultaneous representation of organs and the process of their treatment, etc. In this manner it is possible to simulate the course of a real surgical operation on the screen in (almost) all details.
As a (presumed) solution to the aforementioned problem of lack of information about the orientation of the instruments endoscopic systems have been proposed in which the positions of the instruments are represented on a monitor:
An endoscopic system of the type on which the present invention improves is known from U.S. Pat. No. 5,503,320. The system disclosed in that prior art reference comprises at least one endoscope the objective of which generates an image of the area to be subjected to diagnosis or a surgical operation.
This system is recorded by a video recorder and represented on a monitor. Moreover, at least one instruments such as a pair of scissors, an HF instrument, a clip applicator or the like is provided which is used to perform the treatment or diagnostic process proper. For a determination of the position of the instrument or instruments a position sensing means is provided. For the illustration of the position of the instrument or instruments symbols are displayed on the monitor which represent the positions of the instrument or instruments. The term xe2x80x9cpositionxe2x80x9d is to be understood here to denote the co-ordinates of a particular xe2x80x9cpointxe2x80x9d, as a rule the distal end of the respective instrument.
Similar systems are known from the European Patents EP 0 495 351 B1, EP 0 672 389 A2, the U.S. Pat. Nos. 5,253,647, 5,383,454 and 5,417,210.
By the way, as far as the explanation of all terms not described here in details is concerned, as well as the implementation in engineering terms, i.e. the sensors, assessment and controller units used, etc., explicit reference is made to the aforementioned prior art documents.
In accordance with the present invention the applicant has become aware of the fact that the system known from the U.S. Pat. No. 5,503,320, which discloses the prior art of the claimed general type, entails the disadvantage that merely the position of the distal end is displayed, rather than the orientation of the instrument in space. As a result, it is difficult for the operator to determine the orientation of the instrument on the merely two-dimensional image. The orientation is, however, decisive for the manipulation of the instruments:
When the instrument is moved forward, for instance, the orientation determines the site which the instrument will xe2x80x9chit nextxe2x80x9d.
The present invention is therefore based on the problem of improving an endoscope system of the type described above in such a way that the operator will be able to lead the instruments or instruments more rapidly into the field of vision of the endoscope, and that the operator will furthermore obtain information about the orientation of the instruments.
The invention starts out from an endoscopic system which comprises at least one endoscope the objective of which generates an image of the area to be observed, (at least) one video recorder which records the image of the objective of the endoscope, at least one instrument such as a pair of scissors, an HF instrument, a (technical) manipulator or the like, a position sensing means which detects the position of the instrument or instruments or the endoscope or endoscopes, respectively, and an assessment and control unit to which the output signal of the video recorder and of the position sensing means are applied and which displays an image of the operation field projected by the endoscope objective on a monitor, and furthermore symbols representing the position of the instrument (e.g. surgical instrument).
In accordance with the present invention this endoscopic system is improved by the provisions that the position sensing means comprises sensors which detect not only the position but also the orientation of both the endoscope(s) and the instrument(s), and that the displayed symbols reflect the orientation of the instrument or instruments and possibly of the further endoscopes relative to the displayed image. With endoscopes having a viewing direction which encloses an angle different from 0xc2x0 relative to the longitudinal axis of the endoscope, i.e. so-called inclined-view endoscopes, it is, of course, also possible to fade in some information about their orientation. Moreover, it is also possible to display information about the orientation and particularly the rotational position of the instruments.
The endoscopic instrument set consisting of at least one endoscope with a video recorder system and at least one instrument is hence extended to the effect that each of the endoscope(s) and the instrument(s) comprises a position sensor which detects both the position and the direction or orientation of the instrument or the endoscope, respectively. The local information and the additional positional information are supplied to a processor unit which computes a so-called direction marker which is displayed in the video image on the monitor, preferably in the marginal image area of the endoscopic image or in an (additional) electronic frame which may be disposed around the tube in particular. As will be described still below, the fading-in of the (local and) directional information is performed preferably on when the instruments are not in the field of vision of the endoscope. The directional marker at the margin of the endoscopic image on the monitor is coded in terms of position, direction, and/or spacing from a reference point, e.g. on the line of sight of the endoscope.
In parallel or possibly also as an alternative an acoustic guiding signal may be used as well.
In particular, the symbol represented by the assessment and control unit or the acoustic signal can indicate the direction of entry and the site of entry of the respective instrument into the recorded image. It is particularly preferable to have a symbol represented by the assessment and control unit, which reflects the specific site where the extension of the longitudinal axis of an instrument beyond the image or frame passes through the coverage field cone of the endoscope. This may be realised in particular by the provision that the symbol represented by the assessment and control unit indicates the particular azimuth angle and the direction at or in which the extension of the longitudinal axis of the instrument penetrates the coverage field cone.
The operator receives information about the orientation and the site of the respective instrument in a particularly practice-oriented manner by the fact that the assessment and control unit constructs a plane which is located ahead of the distal end of the endoscope at a definable distance, and that the assessment and control unit represents another symbol indicating the specific site where the extension of the longitudinal axis of the instrument penetrates this plane. As a result, the operator knows whether the instrument will be brought into engagement at the desired site with the tissue to be removed, for example.
This plane may be that plane in particular onto which the objective is focussed, i.e the plane of the image.
It is moreover preferred that the symbol represented by the assessment and control unit will indicate the distance between the distal end of the respective instrument and the image recorded by the objective.
In another embodiment of the invention the symbol indicates whether the axis of the associated instrument does or does not intersect with the coverage field cone of the objective.
In all the aforementioned cases it is moreover expedient that the respective indication is furnished by a different graphic coding of the symbol. The graphic coding may be performed, for instance, in the manner of a bar chart.
The sensors can detect, in particular, also the rotational position of the instruments and/or the endoscope. This is particularly important for so-called inclined-view endoscopes. The assessment and control unit then derives, from the output signal of the sensor associated with an endoscope, the orientation of the optical axis of the endoscope objective. The same applies by way of analogy to instruments performing manipulations which depend on the respective rotational position of the instrument.
In an improved embodiment of the invention the assessment and control unit superimposes the symbols onto the endoscope image on the monitor. It is particularly expedient when the assessment and control unit represents the symbols on the monitor outside the endoscope image proper. To this end a frame may be providedxe2x80x94as has been described above alreadyxe2x80x94which surrounds the display area on the monitor, with the assessment and control unit representing the symbols on this frame. The frame may include light-emitting lighting elements such as miniature lamps or LEDs.
It is preferable in any case that the assessment and control logic represents a symbol only when the associated instrument is not visible in the endoscope imagexe2x80x94as has been described above alreadyxe2x80x94because in such a case the operator will not be strained with the representation of symbols which are, actually speaking, superfluous.
In the system proposed in accordance with the present invention the sensors may be mounted on the proximal or distal ends on the instruments or on the endoscope(s). The endoscope(s) or the instrument(s) may be rigid endoscopes or instruments or flexible endoscopes or instruments. In the case of flexible endoscopes or instruments, respectively, it is preferred that a sensor detects the respective bending.
The inventive system is not only suitable to generate an indication for an operator; in an improved embodiment the assessment and control unit may control a guiding system for the instrument(s). This guiding system may co-operate with the operator in a passive or an active manner, which means that the operator can receive hints as to the manipulation or positioning of instruments or intervene in the handling of the instruments.
It is possible that the guiding system allows for a movement of the distal end of the particular instrument into the endoscope image only, inter alia when the distal end of the instrument or instruments is located beyond the endoscope image. In particular, the guiding system may allow for a manipulation of the respective instrument only when the distal end of this instrument is located in the endoscope image. To this end, the instruments may include a marker which is optically detected by the assessment and control unit by means of the endoscope or endoscopes, respectively, and which serves as point of reference for the guiding system.
On principle, any endoscopes may be employed as endoscope, e.g. conventional rigid or flexible endoscopes on which video units are flange-mounted on the eyepiece funnel. Furthermore, endoscopes may be used where the image recorder of the video unit is disposed on the distal end. The application of stereo endoscopes is particularly preferred which record stereo images. With such endoscopes it is possible that the assessment and control unit defines, by triangulation, the position of a specific point in the endoscope image and selects this point as reference point for the guiding system. In such a case it is preferred that the monitor is a stereo display on which the symbols are displayed in a three-dimensional representation. These three-dimensional symbols may also indicate the point of penetration and the direction of penetration of the respective instrument.
For measuring and hence for a quantitative evaluation moreover a projector means may be provided which projects a two-dimensional pattern onto the area to be subjected to diagnosis or a surgical operation. The assessment and control unit reconstructs then the topography of the area on the basis of the image of the two-dimensional pattern. This pattern may be a network or a dot matrix.